Skip to content

POtel implementation base branch #3152

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 325 commits into
base: master
Choose a base branch
from
Draft

POtel implementation base branch #3152

wants to merge 325 commits into from

Conversation

sl0thentr0py
Copy link
Member

@sl0thentr0py sl0thentr0py commented Jun 10, 2024

Full state of CI: #3744

Contains:

Simple test

import sentry_sdk
from time import sleep

sentry_sdk.init(
    debug=True,
    traces_sample_rate=1.0,
    _experiments={"otel_powered_performance": True},
)

with sentry_sdk.start_span(description="sentry request"):
    sleep(0.1)
    with sentry_sdk.start_span(description="sentry db"):
        sleep(0.5)
        with sentry_sdk.start_span(description="sentry redis"):
            sleep(0.2)
    with sentry_sdk.start_span(description="sentry http"):
        sleep(1)

References

Misc

In OTel, this:

with tracer.start_as_current_span("parent") as parent:
    with tracer.start_span("child1"):
        pass
    with tracer.start_span("child2"):
        pass

is equivalent to

from opentelemetry import trace, context

parent = tracer.start_span("parent")

# Creates a Context object with parent set as current span
ctx = trace.set_span_in_context(parent)

# Set as the implicit current context
token = context.attach(ctx)

# Child will automatically be a child of parent
child1 = tracer.start_span("child1")
child1.end()

# Child will automatically be a child of parent
child2 = tracer.start_span("child2")
child2.end()

# Don't forget to detach or parent will remain the parent above this call stack
context.detach(token)
parent.end()

@sl0thentr0py sl0thentr0py requested a review from sentrivana June 10, 2024 19:00
@sl0thentr0py sl0thentr0py force-pushed the potel-base branch 2 times, most recently from f7f153c to 28effd6 Compare June 11, 2024 11:43
@sl0thentr0py sl0thentr0py force-pushed the potel-base branch 2 times, most recently from 16f9341 to 951477f Compare June 25, 2024 15:16
Copy link

codecov bot commented Jun 26, 2024

Codecov Report

Attention: Patch coverage is 84.27705% with 311 lines in your changes missing coverage. Please review.

Project coverage is 84.38%. Comparing base (18a1104) to head (23eb439).

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
sentry_sdk/integrations/aws_lambda.py 11.11% 48 Missing ⚠️
sentry_sdk/tracing.py 80.41% 37 Missing and 10 partials ⚠️
sentry_sdk/opentelemetry/utils.py 81.93% 25 Missing and 18 partials ⚠️
sentry_sdk/opentelemetry/span_processor.py 81.81% 13 Missing and 17 partials ⚠️
sentry_sdk/integrations/gcp.py 0.00% 27 Missing ⚠️
sentry_sdk/tracing_utils.py 75.80% 10 Missing and 5 partials ⚠️
sentry_sdk/integrations/ray.py 7.14% 13 Missing ⚠️
sentry_sdk/opentelemetry/sampler.py 91.40% 7 Missing and 4 partials ⚠️
sentry_sdk/integrations/tornado.py 75.86% 4 Missing and 3 partials ⚠️
sentry_sdk/scope.py 86.79% 6 Missing and 1 partial ⚠️
... and 23 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3152      +/-   ##
==========================================
+ Coverage   80.23%   84.38%   +4.15%     
==========================================
  Files         142      144       +2     
  Lines       15937    14638    -1299     
  Branches     2727     2329     -398     
==========================================
- Hits        12787    12353     -434     
+ Misses       2273     1558     -715     
+ Partials      877      727     -150     
Files with missing lines Coverage Δ
sentry_sdk/__init__.py 100.00% <100.00%> (ø)
sentry_sdk/_compat.py 84.21% <ø> (-1.84%) ⬇️
sentry_sdk/_init_implementation.py 100.00% <100.00%> (+23.33%) ⬆️
sentry_sdk/ai/monitoring.py 88.57% <100.00%> (+2.46%) ⬆️
sentry_sdk/ai/utils.py 85.00% <100.00%> (+7.72%) ⬆️
sentry_sdk/api.py 96.38% <100.00%> (+16.52%) ⬆️
sentry_sdk/client.py 84.80% <100.00%> (+5.34%) ⬆️
sentry_sdk/consts.py 99.54% <100.00%> (+6.69%) ⬆️
sentry_sdk/debug.py 95.00% <ø> (+3.69%) ⬆️
sentry_sdk/envelope.py 82.98% <100.00%> (+3.08%) ⬆️
... and 84 more

... and 42 files with indirect coverage changes

@sl0thentr0py sl0thentr0py changed the title Skeletons for new POTEL components New POTEL base branch Jul 9, 2024
@sl0thentr0py sl0thentr0py changed the title New POTEL base branch potel implementation base branch Jul 9, 2024
@antonpirker antonpirker changed the title potel implementation base branch POtel implementation base branch Aug 5, 2024
@sentrivana sentrivana removed their request for review August 28, 2024 09:12
antonpirker and others added 18 commits October 8, 2024 14:56
Set the correct span `op` for database spans, instead of the generic `"db"`.
…ts (#3631)

Otel `span.set_attribute()` does not allow `dict` values, thus we change the data to be compatible
`trace_state.add` will never overwrite existing entries -- this is good. However, everytime it bails, it logs a warning, which then shows up in breadcrumbs in tests.

With this commit we explicitly check before adding.
* while sending the event in the final payload, the transaction name
  will be picked up from the attributes `sentry.name` or fallback to the
  `description` when we only have otel instrumentation
* for populating DSC in the head case, we are doing a best attempt
  solution and fetching the transaction name/source from the current and
  isolation scopes
    * NOTE that there are cases where this will be
      inaccurate if we never set the transaction name on the scope in some
      integration, so we will go through and fix those cases separately.
Instead of getting span from ._span_recorder get it from envelope.
- Change `span.set_data` to `span.set_attribute`
- Change `start_transaction`s in the tests to `start_span`s
- Stringify `db.params`, `db.paramstyle`, `db.cursor`
- ❗ Change how adding query source works.
  Previously, we first let the DB span finish and then looked at its start and end timestamps to figure out if the query was slow enough to attach query data.
  
  With the switch to OTel, we can no longer put stuff on a finished span, meaning the slow query span still has to be alive when adding data to it. So instead of looking at the end timestamp of a finished span, we keep the span alive and instead look at current time at the point when the query is finished.
  
  This means the DB span will be longer than the actual query (since it now includes adding query data).
sentrivana and others added 30 commits April 10, 2025 12:02
Moving stuff out of `integrations/opentelemetry/` step by step since
there is no OpenTelemetry integration anymore -- it's part of the core
SDK.

This moves `sentry_sdk/integrations/opentelemetry/sampler.py` ->
`sentry_sdk/opentelemetry/sampler.py`

Ref #3853
This reverts commit 955108e (#3791) and
simply enables `cache_spans` by default.

Co-authored-by: Anton Pirker <[email protected]>
Co-authored-by: Ivana Kellyer <[email protected]>
Closes #4235

---------

Co-authored-by: Ivana Kellyer <[email protected]>
Moving stuff out of `integrations/opentelemetry/` step by step since
there is no OpenTelemetry integration anymore -- it's part of the core
SDK.

This moves `sentry_sdk/integrations/opentelemetry/span_processor.py` ->
`sentry_sdk/opentelemetry/span_processor.py`

Ref #3853

---------

Co-authored-by: Daniel Szoke <[email protected]>
Moving stuff out of `integrations/opentelemetry/` step by step since
there is no OpenTelemetry integration anymore -- it's part of the core
SDK.

This moves `sentry_sdk/integrations/opentelemetry/propagator.py` ->
`sentry_sdk/opentelemetry/propagator.py`

Ref #3853

---------

Co-authored-by: Daniel Szoke <[email protected]>
Moving stuff out of `integrations/opentelemetry/` step by step since
there is no OpenTelemetry integration anymore -- it's part of the core
SDK.

This moves
`sentry_sdk/integrations/opentelemetry/contextvars_context.py` ->
`sentry_sdk/opentelemetry/contextvars_context.py`

Ref #3853
Port of #4278 for POTel.

<!-- Describe your PR here -->

---

Thank you for contributing to `sentry-python`! Please add tests to
validate your changes, and lint your code using `tox -e linters`.

Running the test suite on your PR might require maintainer approval.
Moving stuff out of `integrations/opentelemetry/` step by step since
there is no OpenTelemetry integration anymore -- it's part of the core
SDK. This moves
`sentry_sdk/integrations/opentelemetry/{utils,consts}.py` ->
`sentry_sdk/opentelemetry/{utils,consts}.py`.

There's quite some stuff moving around in this PR in order to remove
circular dependencies that have to do with `tracing.py`:
- `get_span_status_from_http_code` from `tracing.py` is now in
`tracing_utils.py`
- various constants from `tracing.py` are now in `consts.py`

Additionally, Sphinx was unhappy, so tweaked the way we type some things
in `api.py` and `_init_implementation.py`.

I'll possibly follow this up with creating a nice structure for the
tracing files, maybe a common `tracing` directory with `tracing.py`,
`tracing_utils.py` -> `utils.py`, and dedicated `consts.py`.

Ref #3853
Moving stuff out of `integrations/opentelemetry/` step by step since
there is no OpenTelemetry integration anymore -- it's part of the core
SDK.

This moves `sentry_sdk/integrations/opentelemetry/scope.py` ->
`sentry_sdk/opentelemetry/scope.py`

Ref #3853
Moving stuff out of `integrations/opentelemetry/` step by step since
there is no OpenTelemetry integration anymore -- it's part of the core
SDK.

- Moved `sentry_sdk/integrations/opentelemetry/integration.py` ->
`sentry_sdk/opentelemetry/tracing.py`.
- Removed all the experimental autoinstrumentation stuff from
`integration.py`.
- Removed `integrations/opentelemetry/` altogether (there was nothing
left but `__init__.py`, which is now also gone).
- Moved all tests from `tests/integrations/opentelemetry` to
`tests/opentelemetry`.
- Removed the potel/opentelemetry integration test targets from
`tox.ini`. These will now be run as part of the Common test suite.

Ref #3853
Porting stuff from master and other fixes
* add correct `event_level` to new logging tests (on `potel-base`, we
don't capture logging errors by default so this has to be set
explicitly)
* add compat for `start_transaction`
* re-enable an old test

**Note:** This still leaves one failing threading test, will address
that separately
For testing purposes I set the next major version of the SDK.
Currently, this property has type `Any`, but it can now be changed to
`Optional[Span]`

Depends on:
  - #4263
`ThreadingIntegration` can optionally **NOT** propagate scope data to
threads (`propagate_scope=False`). In that case, in POTel we were
wrapping the thread's task in an `isolation_scope()`:

```python
with sentry_sdk.isolation_scope() as scope:
    return _run_old_run_func()
```

But as this forks the currently active isolation scope, the thread
effectively gets all scope data from the parent isolation scope -- so
the scope is actually propagated to the thread, even though it shouldn't
be since `propagate_scope=False`.

~We effectively need some way to give the thread a clear isolation scope
instead. In this PR, I'm just clearing the forked iso scope, but I'm not
sure if this is good enough and if something doesn't need to be done on
the OTel side too.~

~Another option would be to set the iso/current scopes to the initial,
empty iso/current scopes instead, before running the thread's target
function.~

UPDATE: we're just instantiating new scopes now


Another change is that in OTel, the spans in the threads, now without a
parent, automatically get promoted to transactions. (On master they'd
just be orphaned spans, so they wouldn't be taken into account at all.)
We probably need to instruct folks to add `only_if_parent` if they don't
want this to happen.

---------

Co-authored-by: Neel Shah <[email protected]>
With the switch to OTel, the Common test suite is now dependent on an
otel package, so it technically fits the toxgen usecase. By letting
toxgen take care of it, we're making sure we're always testing a good
range of otel versions, including the oldest one (to catch regressions)
and the newest one (to catch incompatibilities early).

Couple things surfaced in terms of incompatibility with older versions:
- Some semantic attributes we're using weren't there from the get go
open-telemetry/opentelemetry-python@495d705.
Changed the code that uses them to handle failure.
- The signature of `span.set_status()` changed at some point
open-telemetry/opentelemetry-python@6e282d2.
Added a compat version of `set_status()` for older otel.

Also included:
- removing the `opentelemetry-experimental` extra (not used anymore)
- ❗ switching to using `opentelemetry-sdk` instead of
`opentelemetry-distro` -- the `distro` only seems to [be setting up some
defaults](https://github.com/open-telemetry/opentelemetry-python-contrib/blob/8390db35ae2062c09d4d74a08d310c7bde1912c4/opentelemetry-distro/src/opentelemetry/distro/__init__.py)
that we're not using


Closes #3241
Closes #4332

---------

Co-authored-by: Daniel Szoke <[email protected]>
Introduce the convention of underscore-prefixed span attributes. These
won't be sent to Sentry and are meant for internal SDK usage.

Changed `flag.count` to internal. Looked through the rest of the attrs
we're setting and that stuff requires a big comprehensive cleanup
altogether to make stuff align with OTel. Didn't touch anything else for
now.

Closes #4329
Revert changing the default of `traces_sample_rate` done in
#4240
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants